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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In Re Application of: 



Date: February 5, 2001 



Anderson 



Serial No.: 08/702,286 



Group Art Unit: 2712 



Filed: August 23, 1996 



Examiner: Harrington, A 



For: A METHOD AND SYSTEM FOR GROUPING IMAGES IN A DIGITAL 
CAMERA 

DECLARATION UNDER 37 C.F.R. 1.131 

I, Eric C. Anderson, hereby declare that: 

1 . I am the inventor of the subject matter recited in the claims of the above-identified 
application. 

2. Prior to December 22, 1995, 1 conceived of the idea of providing a digital camera 

' interface that includes a mark function for allowing a user to mark randomly chosen 
images to create a temporary group of images for collective manipulation. 



3. I conceived of this idea while working for Apple Computer, Inc., in Cupertino, CA, as 
System Architect for Apple's next generation software architecture for image capture 
devices, code-named "FlashPoint". 
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4. Attached as Exhibit A are pages 1 and 3 of a document dated March 18, 1993 that I 
prepared entitled "Adam User Interface State Table," showing the functions assigned 
to function keys displayed on a camera when the camera is placed in various states. 
Page 3, State Number 5, and page 8, Level #1 describe components of a camera 
user interface, including the function of the "Mark" key, which operates in 
accordance with my invention, as described and claimed in the present application. 

5. Attached as Exhibit B are 3 pages of a document dated March 31, 1993, entitled 
"Script of background ID 8642 - "select screen" listing script programming code for 
implementing the select screen of the camera user interface, including the Mark 
function, as described in the document of Exhibit A. 

6. Attached as Exhibit C are pages 1 and 3 of a document dated April 15, 1996 that I 
prepared summarizing inventions conceived for the FlashPoint project. On page 3, ( 
the second paragraph from the bottom is labeled "Method for Creating a Slide Show 

from a Series of Images", which sets forth a description of my invention, as 
described and claimed in the present application. 

7. Attached Exhibit D is a letter dated August 8, 1996 from outside counsel to Eric 
Anderson enclosing a second draft of the present application for review. During the 
course of preparing the present application, the title of the invention had been 
changed to "A Method And System For Grouping Images In A Digital Camera." 
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8, On August 23/1996, tb^ 
invention was filed. 



i hereby ideciare that alt statements made herein; of my own -knowledge are true 



and that all statements made on information 



and belief 



are • believed to be : true; and 



and ; the I ike so made are punishable : by fine or ; imprisonment, or troth, under; Section 
1001 of Title 1 8 of the : United States Code and that : sucti mm \ false ; statement may 
jeopardize the val id ity of the application ; br any patent i issiied thereon i . I • ; 



Eric C; Anderson 



Date'" 
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EXHIBIT A 





Adam User Interface 
State Table 



March 18, 1993 



Eric C. Anderson 



State Number 1 

State Name Power Off 
Display mode off 
Function Keys none 

Power On goto 2: Vlewfinder 



State Number 2 

State Name Vlewfinder 
Display mode Live Video 
Function Keys fash, Expoxure, Focus 

Menu <Flash, Exposure, Focus> (toggle) 
Record Start/Stop audio recording (one recording can be several 
non-contiguous pieces) 

Single mode: recording is appended to single Image. 

Continuous mode: recording and sequence of images is 

combined into a live sequence frame 

Movie mode: recording is soundtrack to movie 
Select If recording, stop. Take image(s) and/or sound in current 

format (single, continuous, movie, Just sound or nothing) 

and goto 5: Select 
Photo If in Single mode, goto 6: Single Photo 

If in Continuous mode, take picture, hold on screen for .2 

seconds. If sound is being recorded, remember time of 

photo for live sequence. Return to live vlewfinder 

If in Movie mode, begin movie and sound capture and goto 

7: Movie Record 
Preferences <Images. Sounds. Other> (toggle) 

Flash Turns flash on/off (flash icon is ON for on, or blinking or 

OFF for off. Blinking flash icon means light level too low for 

highest quality picture) 
Exposure <Darker, Auto, Lighter> (toggle) If you turn it off, it reverts 

to auto exposure. 
Focus Selects one of N focus zones, Zone is highlighted when 

selected (other zones are very dim - barely visible). Zone 

changes from Red highlight to Green mghlight when in 

focus. 

Images <Single, Continuous. Movio 
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Sounds <User, Mono, Stereo 
Other Future 

Single Selects Singk mode [Icon = slide: show Icon in upper right 
viewfinder when menu is up): return to normal viewfinder 
Continuous Selects Continuous mode (icon = movie strip: show icon in 
upper right viewfinder when menu is up); return to normal 
viewfinder 

Movie Selects Movie mode (icon = movie camera: show icon in 
upper right viewfinder when menu is up) 
Goto 3: Movie Size 
Darker Drops exposure 1/2 stop 

Auto Returns to auto setting 
Lighter Increases exposure 1/2 stop 



State Number 
State Name 
Display mode 



Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 
Smaller 
OK 
Larger 



Movie Size 

Show rectangle of current size (1/2 to 1/16) plus prompt 

"Select Movie Image Size". Also show how many frames can 

be stored with currently available resources 

<Smaller, OK, Larger^ 

beep 

beep 

beep 

beep 

beep 

Show next lower size, or beep if at smallest 
Save selection and goto 4; Movie Rate 
Show next larger size, or beep if at largest 



State Number 4 

t State Name Movie Rate 

Display mode Show current rate, plus how many long a movie can be 

created with current resources (show both # of frames plus 
seconds of movie length). Also, prompt "Select Movie Frame 
Rate" 

<Slower. OK. Faster> 
beep 
beep 
beep 
beep 
beep 

select next slower rate or beep if at slowest rate 



Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 
Slower 



2. 
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OK Save selection and goto 2: Viewfinder (note: you could add 
another step here - Movie Quality - if desired) 
Faster select next faster rate or beep if at fastest rate 



State Number 5 

State Name Seleet 
Display mode 9-up display, with current selection highlighted with box. 

Note if this is from power-up and no picture has been taken, 
you are at the beginning of the list 
Function Keys <Mark, View, Goto> 

Menu <Move, Group, Delete> (toggle) 
Record If a Group is selected, Goto 8: Narrate Sequence 

Else. Goto 9: Add Narration 
Select Move selection rectangle around (left, right, up, down) If 

user presses up or down rapidly, go up/down by screenful 
Photo Discard any selection and Goto 2: Viewfinder 
Preferences Discard any selection; Goto 10: ViewBy 

Mark If current selection is not marked, mark and number it 

If current selection is marked, unmark and renumber 
View If a selection has been marked, Goto 11: View Group 
If no selection has been marked: 
If current frame is a single image, Goto 12: View Single 
If current frame is a Group, Goto 11: View Group 
If current frame is a Movie, goto 15: View Movie 
If current frame is sound only, goto 16: View Details 
Goto <First, Cancel, Last> 
Move Goto 13; Move Selection 

Group Create a new frame, link to selected group of frames, and 
store at end of list. Return to Select state with normal 
function keys <Mark, View, Goto> 
Delete Goto 14: Delete Selection 

First Move to first frame in file, restore <Mark, View. Goto> 
Cancel restore <Mark, View, Goto> functions 

Last Move to last frame in file, restore <Mark. View, Goto 



6 

Single Photo 

Hold captured image on-screen. Prompt "press Record to 
complete frame" Show elapsed time of audio, maybe even 
VU meter along bottom 
Function Keys none 

Menu no action (no beep - it will ge recorded) 



State Number 
State Name 
Display mode 
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Record 

Select 
Photo 
Preferences 



Complete audio recording. Current image and recording are 
combined, become current selection. Goto 5: Select 
no action 
no action 
no action 



State Number 
State Name 
Display mode 



Function Keys 
Menu 
Record 
Select 
Photo 

Preferences 



Movie Record 

Uve viewflnder at selected frame rate. A very small green 
bar turns red at bottom of screen, indicating how much of 
available storage is used. If storage runs out, it acts as if 
the user has pressed the Photo button 
none 

no action (don't beep - it will record it) 
no action 
no action 

ends movie recording: movie and sound are combined into 
a frame and become current selection. Goto 5: Select 
no action 



State Number 8 

State Name Narrate Sequence 
Display mode 3-up, where the center image is the current image, and the 
previous and next images can be seen. Note that at the 
beginning, there is no Image on the left, and at the end, 
there is no image on the right. Prompts are as follows: 
"Press Record to begin narration", then "Press Photo to move 
to next image", then "Press Record to end narration", 
f Function Keys <blank, Cancel> 
Menu no action 
Record Starts & Stops narration (see Display Mode). When 

narration is completed, a new frame is generated, which 
includes the list of images, the narration, and the timing. 
Goto 5: Select with this new frame, at the end of the file, 
selected. Group selection is diecarded. 
Select no action 

Photo Sequences images (see Display Mode) 
Preferences no action 

Cancel Goto 5: Select with group still selected 



4 
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State Number 9 

State Name Add Narration 
Display mode 
Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 



State Number 10 

State Name VicwBy 
Display mode 
Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 



State Number 1 1 

State Name View Group 
Display mode 
Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 



State Number 12 

State Name View Single 
Display mode 1-up display: 

Single image: show image full screen 
Function Keys <PreviDus, Details, Next> 
Menu <Previous. Details. Next> 
Record Goto 9: Add Narration 
Select Goto 5: Select 
Photo Goto 2: Viewflnder 
Preferences Future 
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Previous backup to previous frame. 

If previous frame is a single image, Goto 12: View Single 
If previous frame is a Group, Goto 11: View Group 
If previous frame is a Movie, goto 15: View Movie 
If previous frame is sound only, goto 16: View Details 
Details Goto 16: View Details 
Next Backup to previous frame. 

If previous frame is a single image. Goto 12: View Single 
If previous frame is a Group, Goto 11: View Group 
If previous frame is a Movie, goto 15: View Movie 
If previous frame is sound only, goto 16: View Details 



State Number 13 

State Name Move Selection 
Display mode 9-up, with blinking insertion bar 
Function Keys <Insert, Cancel, Goto 
Menu beep 
Record beep 

Select move insertion bar up/down/left/right, including page-up 

and page-down functions 
Photo beep 
Preferences beep 

Insert move selection to insertion point. Goto 5: Select with first 
frame of selection selected. Moves a marked group (and 
unmarks the group) or moves the current selection 
Cancel Goto 5: Select with last marked frame selected. 
Goto <First, Cancel. Last> 

First Goto 5: Select with first frame In file selected (but not 
selected) 

Last Goto 5: Select with last frame in file selected (but not 
marked) 



State Number 14 

State Name Delete Selection 
Display mode 9-up, with last marked frame (and any other marked frames 
that are within the 9-up window) with a blinking red X 
through them. When entering this state, make a special 
WamingBeep 
Function Keys <Confinn, Cancel* 
Menu beep 
Record beep 
Select beep 
Photo beep 
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Preferences 
Confirm 

Cancel 



beep 

Delete selection and Goto 5: Select with next frame after 

last marked frame selected 

Goto 5: Select with last marked frame selected 



State Number 15 

State Name View Movie 
Display mode 
Function Keys 
Menu 
Record 
Select 
Pboto 
Preferences 



State Number 16 

State Name View Details 
Display mode 
Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 



State Number 
State Name 
Display mode 
Function Keys 
Menu 
Record 
Select 
Photo 
Preferences 
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<Mark (UnMark), View (Review), GoTo (UnGoTo)> 

Mark puts a number on the cell, makes a sound. Becomes UnMark. If 
you move selection rectangle to an unmarked cell, function Is Mark. If to 
a marked cell, function Is UnMark. 

View takes you to 1-up display (for image cells). Function is Review for 
sound only cells, and takes you to the Details display. 

GoTo brings up another menu: <First, Marked, Last>. If you press 
either First or Last, you go to the first or last cell in the file, and return 
to the normal functions <Mark, View, UnGoTo where GoTo becomes 
UnGoTo. Any change in the selection or function executed changes 
UnGoTo to GoTo. 

If you press Marked, the functions become <Previous. Next. Done>, 

where you have gone to the last item marked, and you can go backward 
or forward through the marked items. Done gets you out (so does Menu) 

Level #2 

<Group (UnGroup), Move (UnMove), Delete (UnDeleto 

The Group function creates a new cell at the end of the file, and puts all 
the selected (marked) cells Into it, and removes the individual cells from 
the file. The user will not be allowed to make a selection with two labels - 
a warning will be given that two labels are not allowed. The user must 
unlabel one of the cells first. Once a group cell has been made, the user 
can UnGroup until the point where another function is executed. 

Move takes all the marked cells and puts them in the same (sequential) 
location with the first cell marked. It then becomes UnMove, and 
remains until a new function is used. 

Delete deletes the current selection or the marked selection, and 
becomes UnDelete until a new function is used. 

Level #3 

<Duplieate. Label (UnLabel), VlewBy> 

Duplicate allows the user to explicitly make copies of cells - if space 
permits. This allows using same image or sound in several places. There 
are no implicit copies - this gets around the disk space reclaimation 
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# 



problem. Marked cells or current cell is used. No UnDo - user must ' 
reselect and delete. 

Label is shown if current cell is unlabeled, otherwise the function is 
UnLabel. If a selection is marked, this function performs a Group 
followed by a Label function. Labeling takes a user-selected label from a 
list (built-in. but extentable by the user by uploading from his PC) and 
put it on a cell. Only one label per cell is allowed. Pressing Label gives 
you 3x3 grid of available labels in alphabetical order. Use the Select 
keys to select the label. Functions are <Use, Cancel> 

ViewBy allows you to view the file by <date, kind, order>. 



1 - 
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3/31/93 5; 08 pm Script of background id 8642 - "Select^Spreen* 1 

on doMark 

global marks, select, markpos, unmarkno, memory, memory 2 
if visible of bkgnd btn "mark" then 
if marks - 0 then 
put 1 into marks 

put 1 into line select of memory 
show bkgnd btn "1" at markpos 
el$e if marks - 1 then 
put 2 into marks 

put 2 into line select of memory 
show bkgnd btn "2 n at markpos 
else if marks - 2 then 
put 3 into marks 

put 3 into line select of memory 
show bkgnd btn "3" at markpos 
else if marks = 3 then 
beep 

exit doMark 
end if 

put markpos into line select of memory2 
hide bkgnd btn "mark" 
show bkgnd btn "unznark" 
else — DNMARK function 

if line select of memory - 1 then 

hide bkgnd btn "1" 

put 1 into unmarkno 
else if line select of memory - 2 then 

hide bkgnd btn "2" 

put 2 into unmarkno 
else if line select of memory - 3 then 

hide bkgnd btn "3" 

put 3 into unmarkno 
end if 

put (marks - 1) into marks 

put n " into line select of memory 

hide bkgnd btn "unmark" 

show bkgnd btn "majrk n 

if marks > 0 then 

send renumber 
end if 
end if ' 
end doMark 

on Renumber 

global select, marks, memory , markpos, unmarkno, memory2 
if marks = 2 then — do 3 marks to 2 renumber case 
hide bg btn "3" 

if unmarkno =''l then — X23 case - renumber 2 & 3 
repeat with i - 1 to 9 

if line i of memory » "2" then 

show bg btn "l" at (line i of memory2) 
put 1 into line i of memory 
else if line i of memory - "3" then 
show bg btn "2" at (line i of memory2) 
put 2 into line i of memory 
end if 
end repeat 

else if unmarkno = 2 then ■ — 1X3 case - renurriber 3 
repeat with i ■ 1 to 9 — find 3 and renumber to 2 
if line i of memory « "3" then 

show bg btn "2" at (line i of memory2) 
put 2 into line i of memory 
end if 
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3/31/93 5:08 PM Script of background id 8642 - "Select_Screen" 2 

end repeat 

else if unmarkno - 3 then — 12X case - do nothing 
end if 

else — do 2 marks to 1 renumber case 
hide bg btn n 2 n 

if unmarkno - 1 then — X2 case - renumber 2 to 1 
repeat with i - 1 to 9 

if line i of memory ■ "2" then 

show bg btn M l" at (line i of msmory2) 
put 1 into line i of memory 
end if 
end repeat 

else if unmarkno = 2 then — IX case - do nothing 
end if 
end if 
end renurtfber 
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3/31/33 5:09 PM Script of card id 8780 = "Select" 

on opencard 

global storage, level/ marks, memory 

show bkgnd btn "mark" 

hide bkgnd btn "UnMark" 

show bkgnd btn "view" 

hide bkgnd btn "Review" 

show bkgnd btn "goto" 

hide bkgnd btn "UnGoto" 



hide bkgnd btn "group" 
hide bkgnd btn "ungroup" 
hide bkgnd btn "move" 
hide bkgnd btn "unmove" 
hide bkgnd btn "delete" 
hide bkgnd btn "undelete" 



hide bkgnd btn "duplicate 11 
hide bkgnd btn "label" 
hide bkgnd btn "unlabel" 
hide bkgnd btn "viewby" 
put 0 into marks 
put ' Ml into memory 
hide bkgnd btn "1" 
hide bkgnd btn "2" 
hide bkgnd btn "3" 
wait for 1 ticks 
put 1 into level 

if storage ~ "movie_clip" then go card Movie_Clip_Selected 
else if storage = "sound only" then go card SoundjDnly^Card 
else if storage - "continuous^ sound" then go card Multi_Saund_Card 
else if storage = "SinglePhoto&Spund" then go card Ptioto_Sound_Card 
else if storage = "SinglePhoto" then go card Single_Photo_Card 
else if storage * "multiNoSound" then go card Mi;lti_No_Sound_Card 
else if storage = "empty" then go card Last_Item_Card 
end opencard 
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Inventions Summary by Priority 
FlashPoint Project 



Filed Patents 



by Eric C. Anderson 

System Architect 
Image Capture Group 

April 15 1996 



P1311 

PI 366 

P1369 

P1449 

P1450 

PI 560 

P1561 

PI 600 

P1439 

P1451 

PI 873 



Focus Methods and Auto-Macro 
Color Overlay for Focus Assist 
Orientation Sensors In a Digital Cam,™ 
Interrupt System for a Mul&e^ Di^l 

w^° n ^-^ge Process ^ Camera 
Whole Way . Scan AF 

S^^^^S Async Protocol 

MultiMode ROM under SW control 

Battery Utilization with a Flash 

Background Spooling in a Digital Cam^ 

Disk Icon Support for Power 1^ 

-^gf|«o^ihnage Sets (GUI) ~ <4^M<oKt f t>A>^ Ov^p 
• ^^^^^^^^y^ Sn S^^NOWN STATUS/IBM Joint Patent 



AAA Inventions 



«Pp r ™^%^ ^distinctive different 

^f^efaethisASAPl One ^tsu^Tj'T^^ 9 f **™«*™rkel Itis 
NEC (John Moon's) camera doing this sa^ £in£ ^F^Pfon camera or the 
cameru uses Jlash memory, cmda^u^a^Zl^ 1 ^ ^ 0 
know about the NEC camera, bututeelS torfn,£ s P eed c °™Pression I do not 
installed in order to take pictures Squire a removable disk be 



Unified Memory Digital Camera ^C^rp- ) 



processor working memory, and a RAMDtek for S m . age P****** buffers, 
files and processed and compressed^age nies & em P°™y (raw) image 
disk is supported, and typically Is a flash Tdisk or™,??, » nal remova »>le storage 
pis removable storage disk is also "used I for£ teUn ? ma S ne0 <: (hard) disk, 
data, as well as being used for storing™! tem P? ra ry storage of raw image 
disk and the RAMD& a^S^^ fl ^- H» removfble 

maintain a standard director and ^^^^^S^S 



File History Report 



□ Paper number(s) is/are missing from the United 

States Patent and Trademark Office's original copy of the 
file history. 



□ The following page(s) of paper number 

is/are missing from the United States Patent and Trademark 
Office's original copy of the file history. 

□ The following checked item(s) of paper number 

is/are missing from the United States Patent and. Trademark 
Office's original copy of the file history. 

□ PTO-892 Form □ PTO-1449 Form 

□ PTO-948 Form □ Other 
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■EXHIBIT C ^ 

Apple Confidential ^ 



Method for Image File Naming to Resolve Conflicts 

Numbering and Naming Method for Iiria pe* f ^pt^red bv a Digital Cameraf 
his Is a simple system for eliminating name conflicts in a digital camera, by 
using a counter to count the total images captured. This count is used as part 
of the image name. This number Is stored In EEPROM memory to maintain its 
value accurately under all power conditions, including power failures. This 
also eliminates conflicts on ths host when the files are uploaded, since each 
image captured has a unique name. lEric Anderson, Mike Masukawa] 

P?: Method for Image File System Organization and Folder Naming to 

Resolve Conflicts 

Nine (expensive) removalbe storage device is shared between multiple cameras, 
problems can result due to file naming conflicts. This is eliminated by using a 
folder named In part with the camera serial number for the Image files 
generated by a given camera. This ensures no conflict between cameras. This 
includes command set for accessing Images from a given camera from the.host. 
lEric Anderson, Mike Masukawa] 

Camera GUI 

PI 947: Adam GUI: Combining Icon and Image Pane for Grouping 

• Method for Displaying Grouped Images Each image or set of images which 
are grouped by the capture process (such as a burst, a time-lapse, etc) is 
displayed as a single image, along with one. or- more icons to indicate the type. 
Includes sound annotation, single image, sequence of images, movies, slide 
shows, presentations, panaramas, etc. lEric Anderson] 

P?: Adam GUI: Creating a Slide Show 




:ethod for Creating a Slide Show from a seri es of Imag es; This is a method 
or creating a series of. slides using the "mark" function, for the purpose of 
designating a sequential series of images. The group, once marked, can be 
deleted, moved, a soundtrack recorded to make a slide show/presentation, etc. 
Method provides for dynamic rearranging during the marking process. (Eric 
Anderson] 

P?: Adam GUI: Realtime Capture of a Slide Show with Sound 




ethod for Creating a Slide Show with Soundtrack in Real Time using av 
gltal Camera,: This is a method for automatically creating a slide show in a 
single step. (Eric Anderson] 

P?: Adam GUI: Sound Annotation Methodolgy 
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BENM AN COLLINS 8c SAWYER 
ATTORNEYS AT LAW 
620 HANSEN WAY* SUITE A 
Palo alto, Califoiinia 94304 

TELEPHONE (415) 493-4540 
FACSIMILE (415) 493-4549 

INTERNET Mwycr@sirius.com 



August 8, 1996 

VIA FEDERAL EXPRESS 
Eric C. Anderson 
Apple Computer, Inc. 
3585 Monroe Street, MS:68-IC 
Santa Clara, CA 95051 

Re: U.S. Patent Application 

Your Reference No. P1969 
Our Reference No. JAS 553P 

Title: "A METHOD AND SYSTEM FOR GROUPING IMAGES IN A DIGITAL 
CAMERA" 

Dear Eric: 

Please find attached a second draft t>f the above identified patent application 
including claims and informal drawings. Please carefully review the draft patent application 
to insure that the application, including the specification, claims and the hand drafted 
drawings, completely and accurately describe and claim your invention. 

Please provide any additions or changes you may have directly on the corresponding 
pages and fax the pages back to me as soon as possible. 

If you are aware of any additional or more pertinent prior art or other material 
information that should be called to the attention of the Patent Office, please insert 
reference to it directly on the draft application. Please do not hesitate to call should you 
have any questions or comments, 



Very truly yours, 




Stephen G. Sullivan 
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